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Program description 


Voices Literature and Character Education Program ( Voices LACE; 
formerly known as Voices of Love and Freedom and Literacy 
and Values) is a K-12 program that aims to promote positive 
character and citizenship values, literacy skills, and social skills. 
The program contains a curriculum that can be used over any 
length of time. During classroom lessons, students read books 
about such everyday issues as ethnic discrimination, fighting, and 
bullying, and elaborate on central themes through role-playing and 
discussions practiced in school and at home. Emphasis is given 


to promoting caring relationships between teachers and students 
and among students and to connecting the values taught through 
students’ personal stories. Voices LACE may also be implemented 
as a schoolwide improvement program. Optional components of 
the program include schoolwide events and restructuring of school 
organization and practices (establishing student assemblies and 
creating small learning communities), parental involvement (home 
visits and family nights), and community support (joint campaigns 
with supporting organizations and business). 


Research 


One study of Voices LACE met the What Works Clearinghouse 
(WWC) evidence standards with reservations. The study, which 
included 100 sixth- and seventh-grade students from five 


middle schools in one school district in Cambridge, Massachu- 
setts, examined results on students’ knowledge, attitudes, and 
values. 1 



Effectiveness Voices Literature and Character Education Program was found to have no discernible effects on middle school students’ knowledge, 
attitudes, and values. 





Behavior 


Knowledge, attitudes, and values 


Academic achievement 


Rating of effectiveness 


Not reported 


No discernible effects 


Not reported 


Improvement index 2 


Not reported 


Average: +6 percentile points 
Range: +6 percentile points 


Not reported 



1. The evidence presented in this report is based on the available research. Findings and conclusions may change as new research becomes available. The 
findings in the one study meeting standards with reservations pertain to the Voice LACE curriculum rather than the whole school improvement model. 

2. These numbers show the average and range of improvement indices for all findings in the one study reviewed. 
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Developer and contact 

Zaner-Bloser/Voices Programs. 217 California Street Newton, 

MA 02458. Web: www.zaner-bloser.com . Email addresses 
of sales consultants vary by state and are available on the 
developer’s website. Telephone: 617-964-3503. 

Scope of use 

Voices LACE, established in 1992, has been adopted by more 
than 750 urban, suburban, and rural schools around the country 
with varying demographic characteristics. The Voices LACE 
program may have changed since the studies were conducted. 
The WWC recommends asking the developer for information 
about the most current version of this program and taking into 
account that student demographics and school context may 
affect outcomes. 

Teaching 

Voices LACE uses children’s literature to help students 
develop positive character and civic values, multicultural 
understanding, literacy skills, and social skills. The program 
emphaszes promoting caring relationships between teachers 
and students and among students and to connecting the 
values taught through students’ personal stories. The cur- 
riculum was designed for grades K-12, with developmental^ 
appropriate lesson duration and content. The core curriculum 
was designed to be integrated into existing reading, writing, 
language arts, health, or social studies classes. For each 
grade level, the curriculum includes multicultural stories and 
written teacher resources that provide suggestions on how to 
integrate the teaching of character education and violence pre- 
vention into existing classes. Using this curriculum, students 
explore and practice such core values as honesty, fairness, 
perseverance, hard work, courage, loyalty, trust, responsibility, 
and self-discipline. An alternative curriculum for grades K-3, 
Voices Reading, combines elements of a core reading program 
with multicultural literature organized by themes. For grades 
K-3, there are picture books that the teacher reads aloud to 



the students. For grades 3-6, there are three to five books per 
theme, including at least one chapter book that can be read 
aloud or used for shared reading. There are student books to 
practice social and literacy skills. 

Schools may choose to adopt Voices School Design, which 
may include schoolwide practices, parental involvement, and 
community support. Schools adopting Voices School Design 
incorporate core values into their mission statements and work 
to create supportive classroom and school environments. 
Schoolwide practices may include staff professional develop- 
ment, student assemblies and displays of student work inside 
and outside the classroom, a counseling program, and small 
learning communities. The parental involvement component may 
include potluck dinners or breakfasts with families, family nights, 
a parent involvement team, individual parent-teacher confer- 
ences, home visits, and homework support by parents. The 
community support component may include joint campaigns 
with supporting organizations and business and an after-school 
reading program. 

The developer provides institutes to introduce the school to 
the Voices LACE curricula and Voices School Design. Profes- 
sional development includes one-day introductory training and 
multiple days of on-site coaching and modeling of Voices LACE 
lessons in classrooms. 

Cost 

A teacher resource guide costs $19.99. There are approximately 
10 teacher resource guides with accompanying trade books 
per grade level for grades K-4, 6 teacher guides with books 
per grade level for grades 5-8, and 2 teacher resource guides 
with books per grade level for grades 9-12. The accompanying 
children’s story books range in cost from $4-$8 for paperback 
titles and $10-$20 for hardcovers. Middle and high school 
books are available only in paperback. Multiple copies of the 
literature (one copy per student) are necessary to implement the 
program in grades 6-12. Teacher training costs $1,400 a day 
plus expenses. 
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Research 



Effectiveness 



The WWC found Voices 
Literature and Character 
Education Program to 
have no discernible 
effects on knowledge, 
attitudes, and values 5 
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Eleven studies reviewed by the WWC investigated the effects 
of the Voices Literature and Character Education Program. One 
study (Demetriades-Guyette, 2002) of the impact on middle 
school students of the Voices LACE curriculum in a 12-week 
implementation 3 (Literacy and Values) was a quasi-experimental 
design that met WWC evidence standards with reservations. The 
other 10 studies did not meet WWC evidence standards. 



The Demetriades-Guyette study included 100 students 
from five middle schools in one school district in Cambridge, 
Massachusetts. Outcomes for students receiving Voices 
LACE were compared with those for students in schools that 
did not use this program. The study focused on Voices LACE 
as implemented in classrooms rather than as a schoolwide 
intervention. 



Findings 

The WWC review of character education addresses student 
outcomes in three domains: behavior; knowledge, attitudes, and 
values; and academic achievement. 

Knowledge, attitudes, and values. Demetriades-Guyette 
examined two outcomes in this domain but did not find statisti- 
cally significant differences between the intervention and com- 
parison groups. These findings were confirmed by the WWC. 
The average effect size across the two outcomes in this domain 
was neither statistically significant nor substantively important 
using WWC criteria. 4 



Rating of effectiveness 

The WWC rates interventions as positive, potentially positive, 
mixed, no discernible effects, potentially negative, or negative. 
The rating of effectiveness takes into account four factors: the 
quality of the research design, the statistical significance of the 
findings (as calculated by the WWC), the size of the differences 
between participants in the intervention condition and the com- 
parison condition, and the consistency of the findings across 
studies (see the WWC Intervention Rating Scheme) . 



Improvement index 

For each outcome domain, the WWC computed an improve- 
ment index based on the effect size (see the WWC Improve- 
ment Index Technical Paper). The improvement index repre- 
sents the difference between the percentile rank of the average 
student in the intervention condition versus the percentile rank 
of the average student in the comparison condition. Unlike 



the rating of effectiveness, the improvement index is entirely 
based on the size of the effect, regardless of the statistical 
significance of the effect, the study design, or the analysis. The 
improvement index can take on values between -50 and +50, 
with positive numbers denoting favorable results. The average 
improvement index for knowledge, attitudes, and values is +6 
percentile points. 



3. The developer notes that there is no required minimum length and intensity of program delivery. But both the developer and the study author recommend 
a long-term implementation using multiple program components to substantially affect students’ thinking and behavior. Currently, there is no research 
evidence that met WWC standards and examined a long-term full implementation of the Voices LACE program. 

4. The level of statistical significance was calculated by the WWC and corrects for clustering within classrooms or schools and for multiple comparisons. For an 
explanation, see the WWC Tutorial on Mismatch . See the WWC Intervention Rating Scheme for the formulas the WWC used to calculate statistical significance. 
In the case of the Voices report, no corrections for clustering or multiple comparisons were needed. 

5. The one study that met WWC standards with reservations (Demetriades-Guyette, 2002) also examined student outcomes in the behavior domain. Student 
behavior outcome measures were not reviewed because of incomplete statistical information. 
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The WWC found Voices 
Literature and Character 
Education Program to have 
no discernible effects 
on knowledge, attitudes, 
and values 3 (continued) 



Summary 

The WWC reviewed 11 studies on Voices LACE. One study 
met WWC standards with reservations. This study used a 
quasi-experimental design that assessed middle school student 
outcomes in the knowledge, attitudes, and values domain. Both 
outcomes examined in this domain were not statistically sig- 
nificant. When the WWC aggregated the results in this domain 



the average effect size was neither statistically significant nor 
substantively important. The WWC rated the program as having 
no discernible effects on knowledge, attitudes, and values. 
Character education, an evolving field, is beginning to establish 
a research base. The evidence presented in this report is limited 
and may change as new research emerges. 
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school-based multicultural literature program. Dissertation 
Abstracts International, 63(05B), 2615. (UMI No. 3052695) 

Did not meet WWC evidence screens 

Bol, L., Ross, S. M., Nunnery, J. A., & Alberg, M. S. (2002). A 
comparison of teachers’ assessment practices in school 
restructuring models by year of implementation. Journal of 
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Calaway, F. (2001). Evaluation of the comprehensive school 
reform models in the Memphis City Schools. Memphis, TN: 
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Daiute, C., Stern, R., & Lelutiu-Weinberger, C. (2002a). Negotiating 
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73). Washington, DC: American Psychological Association. 6 
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Ross, S. M., & Lowther, D. L. (2003). Impacts of the Co-nect 
school reform design on classroom instruction, school cli- 
mate, and student achievement in inner-city schools. Journal 
of Education for Students Placed at Risk, 8(2), 21 5-246. 6 

Selman, R. L. (1998). Research and practice in the promotion of 
intergroup relations in schools. Boston, MA: Harvard Univer- 
sity, The Group for the Study of Interpersonal Development. 7 

Suriani M. M., & Wasson, R. (1994, November). Assessment of an 
intervention addressing literacy and ethics. Paper presented 
at the annual meeting of the Mid-South Educational Research 
Association, Nashville, TN. 6 

Wasson, R., Anderson, R., Huffman, S., & Suriani, M. (1995). The 
implementation of Voices Literature and Character Education 
Program: A summative report. Memphis, TN: The University of 
Memphis, Center for Research in Educational Policy. 6 

Wasson, R., Huffman, S. B., Burr-McNeal, B., & Kenney, G. 
(1994). Development and piloting of the prosocial attitude 
bank. Memphis, TN: The University of Memphis, Center for 
Research in Educational Policy. 8 



For more information about specific studies and WWC calculations, please see the WWC Voices Literature and 
Character Education Program Technical Appendices . 



6. Does not use a strong causal design: the study did not use a comparison group. 

7. Does not use a quantitative design: this study used a qualitative approach to report findings. 

8. Does not use a strong causal design: the study, which used a quasi-experimental design, did not establish that the comparison group was equivalent to 
the treatment group at the baseline. 
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Appendix 



Appendix A1 


Study characteristics: Demetriades-Guyette, 2002 (quasi-experimental design) 


Characteristic 


Description 


Study citation 


Demetriades-Guyette, A. (2002). Patterns of change in the social-cognitive development of middle school children following a school-based multicultural literature program. 
Dissertation Abstracts International, 63(05B), 2615. (UMI No. 3052695) 


Participants 


The study included 100 sixth- and seventh-grade students from five middle schools. About 50% of the sample was female. A higher percentage of minority students were in 
the intervention group (77%) than in the comparison group (57%). The largest minority group in both the intervention (35%) and comparison conditions (20%) defined them- 
selves as multiracial. The second largest minority group was African-American students in the intervention group (14%) and Asian students in the comparison group (13%). 


Setting 


Both the intervention and comparison schools were part of Cambridge Public Schools, Massachusetts. 


Intervention 


The program consisted of a 12-week literature-based curriculum. The program was co-taught by the regular classroom teachers and a staff developer from the Cambridge 
Youth Guidance Center. 


Comparison 


The comparison schools were drawn from the same school districts as the intervention schools and were matched on demographic characteristics. Comparison schools did 
not implement the Voices Literature and Character Education Program but intended to implement the program after the end of the study. 


Primary outcomes 
and measurement 


The study investigated student outcomes in the knowledge, attitudes, and values domain using the GSID Relationship Questionnaire (REL-Q). This student survey included five 
subscales measuring interpersonal understanding and interpersonal skills. (See Appendix A2 for more detailed descriptions of the outcome measures.) 1 


Teacher training 


Teachers participated in a three-day training prior to program implementation. 



1 . The study also used a behavior count questionnaire assessing frequencies of negative and prosocial behaviors. The study author reported no statistically significant effects for these student outcomes. The WWC could not review these 
measures because means and standard deviations were not available for review. 
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Appendix A2 Outcome measures in the knowledge, attitudes, and values domain 



Outcome measure 


Description 


The GSID Relationship 
Questionnaire: maturity (best 
response score) scale 


GSID Relationship Questionnaire (as cited in Demetriades-Guyette, 2002): Relationship Maturity, scored by “best response.” Total score on a 24-item survey (Rel-Q) 
developed by the Group for the Study of Interpersonal Development (GSID). The survey consists of five subscales. Two subscales represent interpersonal understanding 
(understanding and perspective-taking), two subscales represent interpersonal skills (hypothetical and real-life interpersonal negotiation), and the fifth Rel-Q subscale 
represents personal meaning awareness. The best response score is based on the student's choice of the best response of four possible responses to each question. 


The GSID Relationship 
Questionnaire: maturity 
(item rating score) scale 


GSID Relationship Questionnaire (as cited in Demetriades-Guyette, 2002): Relationship Maturity, scored by “item rating.” Total score on a 24-item survey (Rel-Q) 
developed by the Group for the Study of Interpersonal Development (GSID). The survey consists of five subscales. Two subscales represent interpersonal understanding 
(understanding and perspective-taking), two subscales represent interpersonal skills (hypothetical and real-life interpersonal negotiation), and the fifth Rel-Q subscale 
represents personal meaning awareness. The response rating score is based on the student's assignment of “poor,” “average,” “good,” and “excellent” to each of four 
responses to each question. 
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Appendix A3 Summary of study findings included in the rating for the knowledge, attitudes, and values domain 1 









Author’s findings from the study 














Mean outcome 














(Standard deviation 2 ) 




WWC calculations 








Sample size 


Voices LACE 


Comparison 


Mean difference 4 


Statistical 






Study 


(students/ 


group 3 


group 


(column 1- 


significance 6 


Improvement 


Outcome measure 


sample 


schools) 


(column 1) 


(column 2) 


column 2) 


Effect size 5 (at a = 0.05) 


index 7 



Demetriades-Guyette, 2002 (quasi-experimental design) 



Relationship maturity 


Grades 6-7 98 students/ 


2.10 


2.05 


0.05 


0.16 


ns 


+6 


(best response score) 


5 schools 


(0.31) 


(0.31) 










Relationship maturity 


Grades 6-7 98 students/ 


2.09 


2.06 


0.03 


0.16 


ns 


+6 


(item rating score) 


5 schools 


(0.18) 


(0.15) 










Domain average 8 for knowledge, attitudes, and values 








0.16 


ns 


+6 



ns = not statistically significant 



1. This appendix reports findings considered for the effectiveness rating and the improvement index. Findings on the individual item level are presented in Appendix A4. 

2. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. 

3. The WWC requested and received means and standard deviations for all outcomes in the knowledge, attitudes, and values domain, because they were not reported in the paper. 

4. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 

5. For an explanation of the effect size calculation, please see the WWC Technical Working Paper on Effect Size . 

6. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between groups. The level of statistical significance was calculated by the WWC and corrects for clustering 
within classrooms or schools and for multiple comparisons. For an explanation see the WWC Tutorial on Mismatch . See the WWC Intervention Rating Scheme for the formulas the WWC used to calculate statistical significance. In the 
case of the Voices report, no corrections for clustering or multiple comparisons were needed. 

7. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values 
between -50 and +50, with positive numbers denoting favorable results. 

8. This row provides the study average, which is also the domain average in this case. The WWC-computed domain average effect size is a simple average rounded to two decimal places. The domain improvement index is calculated from 
the average effect size. 
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Appendix A4 Summary of detailed study findings for the knowledge, attitudes, and values domain 1 









Author’s findings from the study 














Mean outcome 














(Standard deviation 2 ) 




WWC calculations 








Sample size 


Voices LACE 


Comparison 


Mean difference 4 


Statistical 






Study 


(students/ 


group 3 


group 


(column 1- 


significance 6 


Improvement 


Outcome measure 


sample 


schools) 


(column 1) 


(column 2) 


column 2) 


Effect size 5 (at a = 0.05) 


index 7 



Demetriades-Guyette, 2002 (quasi-experimental design) 



Interpersonal understanding 
(best choice score) 


Grades 6-7 


98 students/ 
6 schools 


1.95 

(0.43) 


1.91 

(0.40) 


0.04 


0.09 


ns 


0.04 


Hypothetical negotiation 


Grades 6-7 


98 students/ 


2.00 


2.04 


-0.04 


-0.10 


ns 


-0.04 


(best response score) 




6 schools 


(0.58) 


(0.07) 










Real life negotiation 


Grades 6-7 


98 students/ 


2.13 


2.16 


-0.03 


-0.05 


ns 


-0.02 


(best response score) 




6 schools 


(0.60) 


(0.64) 










Personal meaning 


Grades 6-7 


98 students/ 


2.12 


1.95 


0.17 


0.34 


ns 


0.13 


(best response score) 




6 schools 


(0.41) 


(0.57) 










Perspective coordination 


Grades 6-7 


98 students/ 


2.22 


2.05 


0.17 


0.36 


ns 


0.14 


(best response score) 




6 schools 


(0.49) 


(0.45) 










Interpersonal understanding 


Grades 6-7 


98 students/ 


2.03 


2.04 


0.00 


-0.02 


ns 


-0.01 


(item rating score) 




6 schools 


(0.22) 


(0.19) 










Hypothetical negotiation 


Grades 6-7 


98 students/ 


2.15 


2.14 


0.01 


0.02 


ns 


0.01 


(item rating score) 




6 schools 


(0.28) 


(0.24) 










Real life negotiation 


Grades 6-7 


98 students/ 


2.13 


2.16 


-0.03 


-0.06 


ns 


-0.02 


(item rating score) 




6 schools 


(0.25) 


(0.64) 










Personal meaning 


Grades 6-7 


98 students/ 


2.01 


1.97 


0.03 


0.16 


ns 


0.07 


(item rating score) 




6 schools 


(0.23) 


(0.18) 










Perspective coordination 


Grades 6-7 


98 students/ 


2.17 


2.10 


0.07 


0.30 


ns 


0.12 


(item rating score) 




6 schools 


(0.26) 


(0.21) 











ns = not statistically significant 

1. This appendix presents item-level findings for measures that fall in the knowledge, attitudes, and values domain. Aggregated scale scores were used for rating purposes and are presented in Appendix A3. 

2. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. 

3. The WWC requested and received means and standard deviations for all outcomes in the knowledge, attitudes, and values domain, because they were not reported in the paper. 

4. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 

5. For an explanation of the effect size calculation, please see the WWC Technical Working Paper on Effect Size . 

6. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between groups. The level of statistical significance was calculated by the WWC and corrects for clustering 
within classrooms or schools. For an explanation see the WWC Tutorial on Mismatch . See the WWC Intervention Rating Scheme for the formulas the WWC used to calculate statistical significance. In the case of the Voices report, no cor- 
rections for clustering were needed. 

7. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values 
between -50 and +50, with positive numbers denoting favorable results. 
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Appendix A5 Rating for the knowledge, attitudes, and values domain 



The WWC rates interventions as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative. 1 

For the outcome domain of knowledge, attitudes, and values, the WWC rated Voices Literature and Character Education Program as having no discernible effects. It did 
not meet the criteria for positive effects, because it only had one study. In addition, it did not meet the criteria for other ratings (potentially positive effects, mixed effects, 
potentially negative effects, and negative effects) because the single study that met WWC standards did not show statistically significant or substantively important effects. 

Rating received 

No discernible effects: No affirmative evidence of effects. 

• Criterion 1 : None of the studies shows a statistically significant or substantively important effect, either positive or negative. 

Met. The WWC analysis found no statistically significant or substantively important effects in this domain. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1 : Two or more studies showing statistically significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. Voices Literature and Character Education Program had only one evaluation study meeting WWC evidence standards with reservations 
that reported findings on knowledge, attitudes, and values. 

• Criterion 2: No studies showing statistically significant or substantively important negative effects. 

Met. The WWC analysis found no statistically significant or substantively important negative effects In this domain 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1 : At least one study showing a statistically significant or substantively important positive effect, thus qualifying as a positive effect. 

Not met. The WWC analysis found no statistically significant or substantively important positive effects in this domain. 

• Criterion 2: No studies showing a statistically significant or substantively important negative effect. Fewer or the same number of studies showing indeterminate 
effects than showing statistically significant or substantively important positive effects. 

Not met. The WWC analysis found no statistically significant or substantively important negative effects in this domain. Because one study showed 
indeterminate effects and no studies showed statistically significant or substantively important positive effects, Voices Literature and Character 
Education Program did not meet this criterion. 

(continued) 
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Appendix A5 Rating for the knowledge, attitudes, and values domain (continued) 



Mixed effects: Evidence of both positive and negative effects. 

• Criterion 1 : At least one study showing a statistically significant or substantively important positive effect. At least one study showing a statistically significant or 
substantively important negative effect, but no more such studies than the number showing a statistically significant or substantively important positive effect. 

Not met. The one study that met WWC standards with reservations did not show statistically significant or substantively important positive or 
negative effects in this domain. 

OR 

• Criterion 2: At least one study showing a statistically significant or substantively important effect, and more studies showing an indeterminate effect than showing 
a statistically significant or substantively important effect. 

Not met. The one study that met WWC standards with reservations did not show statistically significant or substantively important effects in this 
domain. 

Potentially negative effects: Evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1 : At least one study showing a statistically significant or substantively important negative effect. 

Not met. The one study that met WWC standards with reservations did not show statistically significant or substantively important negative effects 
in this domain. 

• Criterion 2: No studies showing a statistically significant or substantively important positive effect, or more studies showing statistically significant or substantively 
important negative effects than showing statistically significant or substantively important positive effects. 

Not met. The one study that met WWC standards with reservations did not show statistically significant or substantively important negative effects 
in this domain. 

Negative Effects: Strong evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1 : Two or more studies showing statistically significant negative effects, at least one of which is based on a strong design. 

Not met. The one study that met WWC standards with reservations did not show statistically significant negative effects in this domain. 

• Criterion 2: No studies showing statistically significant or substantively important positive effects. 

Met. The one study that met WWC standards with reservations did not show statistically significant or substantively important positive effects in 
this domain. 

1 . For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain level effect. The WWC also considers the size of the domain level effect for ratings of potentially positive effects. See the WWC 
Intervention Rating Scheme for a complete description. 
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